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ABSTRACT 

A study examined the results of an administration of 
a series of theoretically based prototype tests to 857 high school 
students in California, New York, and Wisconsin. By revising the 
existing framework of a prior study, tests wero devised which 
attempted to measure three interrelated aspects of school literature: 
background knowledge, the ability to read and to. articulate a written 
response, and preference (concluding aesthetic judgment of texts and 
general habits and beliefs concerning literature.) Results indicated 
that the three subdomains of knowledge, practice, and habits or 
preferences were more distinct than related, tending to 
interpenetrate each other to a lesser extent than curriculum planners 
and theorists might predict. Results also showed that knowledge 
affected practice but was hot a substitute for it, the same being 
true for interests and qualities of reading. Results further 
demonstrated a distinction between demonstrating understanding of a 
text and articulating a sustained response to a text, and that 
reading and answering someone else's questions and writing out one's 
own understanding of a text were related but hardly equivalent. 
(Eight tables of data and 21 references are included; 4 appendixes 
provide a note on item classification, sample measures of the domain 
of literature learning, scoring guidelines and scale, and 
supplemental tables.) (PRA) 
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PROTOTYPE MEASURES OF THE DOMAIN OF LEARNING IN LITERATURE 



Hongni Li 
Alan C Purves 
Margaret Shirk 
UnivCTsity at Albany 
State University of New York 



One of the major concerns about literature education in the United States of America is that 
concerning its goals and objective, panicularly as they are reflected in assessment. In raising his Hrst 
concerns about cultural lit^cy, Hirsch (1983) claimed that the standanls of cultural literacy were set by the 
college admissions toting programs. Other critics have raised questions about how literature is tested in 
state and national as^ssments and the effects of these tests on the naticmal curriculum ( Applebee, Langer, 
and Mullis, 1987; Ravitch and Finn, 1987). A recent review of existing United States literature tests 
(B^y. DeMilo and Purves. 1989) showed that« in general, literature as a r^hool subject is not tested- 
nei her the knowledge nw the skills and abilities relied to re^ng a text. This problem appeared to stem in 
l^n from the limitations of the multiple-choice test. 

The scholarty hisvxry of the testing of learning in literature* however, has shown a great deal of 
variety and ingenuity in measurement (Purves, 1971; Purves & Beach. 1972, Cooper 1985). Although 
many approaches to measuring students* understanding and appr^iation of literature and their altitudes and 
beliefs towards literature have been studied* they have not been employed in any attempt to make a 
com(»ehensive assessment of student learning. In most tests, measures of knowledge are divorced from 
meas'ires of critical reading, and both separated from measures of teamed attitudes and beliefs. Some tests, 
such as the Graduate Record Examination, bring together disparate aspects of student learning, but the 
relationships among these aspects are not explored. The question of what might provide a comprehensive 
assessment of student learning of literature has been ignored. To remedy this gap in our understanding of 
student learning and achievement in relation to the intended literature curriculum as stated in goals and 
objectives, the present series of studies has been initiated. 

This report is the second in a series of two. The fir^ (Purves, Li and Shirk, 1990) began with the 
establishment of a thecMetical depiction of the domain of literature teaming and then reported on a series of 
studies to probe the interrelationships among the various subdomains. We will review the theoretical 
discussion in this report and then examine the results of an ^ministration of a series of theoretically based 
prototype tests to secondary school students in California. New York, and Wisconsin. 



I. Background 

The Domain of School Liierature 

In curricula and syllabi, school literature is usually seen as one of the language arts, which have 
often been deflned in terms of reading, writing, speaking and listening. Since literature involve texts that 
people read or write, and since when students read literature they often write about whnt they have read, 
literature is often seen simply as a subset of reading and writing, with an occasional nod to speaking and 
listening (neither of which is usually tested; and they will not be in this study). But those who take a 
serious interest in literature as a school subject are uneasy with this definition. They become more uneasy 
when they lock at the world of tests and see that literature is simply a vehicle for tests of reading 
comprehension or for measures of writing skill or proficiency (Brody, DeMilo and Purves, 1989). There 
seems to be the need for something more. To define the literature curriculum as simply a subset oi reading 
and writing neglects a number of the acts that go on within the activity of literature education. 

Some would define literature as a %hoot subject that has its own corpus of knowledge. Recently, 
this corpus of knowledge has come to be included under the heading '^cultural literacy."(Hirsch. 1983, 
Ravitch and Finn* 1987). Narrowly defined, the corpus refers to the names associated with a particular set 
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of texts: auUws. characters, plots, and themes. But it might well be brazened to include such matters as 
critical tmns like metaphc^ and simile, as well as genres, styles of writing, and whole critical approaches. 

There is another group that would see literature as something that is read differently than other 
kinds of infoimational texts. Louise Rosenblatt ( 1978) calls this kind of reading aesthetic and opposes it to 
the reading that one does with infwmational texts such as those of social studies and science (see also 
Langer, 1989). From this ^{»irach, one sees that a i^i of literature education is the development of 
Inferences: habits of mind in reading and ^vriting. In Edition, lit^ture education is supposed to develop 
something called ''taste'* the tove of ''good literature.'' so that liiemmre educatioi goes beyoml reading and 
writing in the inculcation of specific sets of preferences and tiabits of reading and writing about that 
particular body of texts which is called lii^ture. 

Practically speaking, each of these advocates of a particular appo^h to literature instruction is 
partially correct It would appear from studies of >^rious curricula in literature at all levels of edircation that 
a part df learning is learning to read and write about what is reml. a part is acquiring some knowledge of the 
nature and history of literature, and a part is devek^ng ai^ropriate habits of mind and beliefs about 
literaitire (Purves. 197 h 1973. Wellek and Warren. 1976. Hawisher, 1990). We would then argue that, for 
the purposes of measurement, the domain of school literaiiuie can best be divided into three interrelated 
aspects: knowl^ge. practice, and preference. The interrelationships are complex in that one uses knowledge 
in the various acts that constitute practice and preferences, and practices and inferences can hrve their 
influence on knowledge. Some attempts to measure these aspects together have been made, most notably in 
the National Assessment of Literature in 1970 and the lEA study of literature ^ucation in ten countries 
(Purves. 1973). Cooper ( 1985) also reports on attempts to create a domain-based asscssmeni. 

One can hypothesize that the intersections among the aspects can best \k seen (at Kast in the 
United States pedagogical tradition. Purves, 1971) in the relationship between knowledge of critical terms 
and the practice of writing about texts, and in the practice of reading and writing and aesthetic judgments. 
One could argue that these judgments, baspd on actual reading, serve to steer the ?wamg interests and habits 
of people. At the same time, one can separate them for the purposes of nesting and curriculum planning. We 
may schematize the three subdomains as follows (Table I): 
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Table 1 
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LITEkRTURE 
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PREFERENCE 
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Genres 
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Cultural 








Person- 






Allusion 




Styles 




alize 




General- 














ize 






Critical 




Inter- 




across 






Terms 




pret 




works 




Aesthetic 



Evaluate 
Select 



Habits 
Reading 
Critical 

Value 



If we accept this depiction of the domain, we see that one major question facing those concerned 
with test generation is to detennine what sorts and ccmibinations of act texts, and types of itons might 
hesi be suited to meastirc a stu(tent*s performance within and across the domain. The issue of item type is 
central to this study, for there is concern throughout the profession that the standard item type--the 
multiple-choice or matching question-is too limited (Brody, DeMilo, & Purves, 1989). A review of 
existing tests and measures in this c^ntry and overseas produced the classification of item types found in 
Appendix I. Against this background* the present smdies were designed. 

Results of the First Study 

Treating the study as a pilot-testing 'Exercise* we could reach some clear conclusions as to what 
might make a good comprehensive test packnge. Tte results tend to validate the mo(kl that knowl^ge. 
practice* and preference are relau^ but not highly interrelated aspects of the construct of literature learning. 
A comprehensi w m^ure of student performance, therefore, will address each of th^ uuee ar^. Within the 
knowledge domain, textual knowledge and knowledge of critical terms are distinct, particularly in their 
relationship to the practice of reading and responding. Ahhough in pedagogy f uch knowledge is inte^ed 
into pr^tice. for assessment purposes it is import^t to see if students have the knowledge as well as 
whether they use it. Within the domain of practice, more than one passage is needed to get some estimate of 
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a student's pcrformaiKc across text types. It makes little difference whether one uses open-ended or 
irultiple-chotce questions, but one can argue on other grounds that open-ended questions present more of a 
challenge U> students than multiple-choke questions (Hansson, 1990), and are a more exacting measuie of 
the ability to read and shape a le^xm^ to what is read 

It is also clear that an extemted response is also desuable, but otter studies argue that the {during 
of the question might be such as to allow the student some preparation for the setting forth of a fully 
articulated composition. A stark qu^cm is less desirable than a que5;ion ihat builds upon another sort of 
task, one that gets the snident to consider the text in question (Hansf ^. 1990). A combination of multiple- 
choice ami essay or scale and essay forms tt«e optimum measure. 

In the realm of preference, it is important \o separate determining the student's criteria forjudging a 
text from the actual judgment. It is also important tc get a depiction of the general attitudes toward 
literature including censorship, since these are related to cognitive performance (whei(^r in an antecedent or 
consequent role remains unclear). 

Based on these fmdings. we derived a set of specifications for a comprehensive assessment of 
stucfent learning in literamre that includes the following: 

1 . Measures of t^kground knowledge»terminology and cultural information: the.se may 
include matching and supplying or generating items. 

2. Measures of the ability to read and to articulate a written response to at least two texts 
that differ in genre, the me^ur^ to include both supplying and constructing items, with 
the latto* taking the fonn extended discourse. 

3. Measures of preference iiKluding aesthetic judgment, of specific texts and general habits 
and beliefs coiKeming literature and its place in the 'vorld. 

Such an assessment gives a more comprehensive picture of student learning and also of program 
effectiveness than does a measure of any one taken alone. A r^ent study (Ho, 1988) that used a more 
comprehensive measure showed that a complex measure served best to validate a curriculum or a model of 
instniction. If the intention of the instruction is U) make classroom exploration of literature more open and 
to use more ^^real*" and thought-provc^ng que^oning than ncmnal inaction, its validation must include 
measttres of both practice and preference. The develc^ment and trial of an assessment package, that can serve 
as a model of assessment at the state or disuict level and at the cla^room level* forms the basis of the 
present sttidy. The package is not ctesigned to measi^ individual sndent perfomiance. 



IL The Present Study 

Since the first shidy produced a number of reliabtr, and tested measwes. the research team composed 
the new set of tests by revising the items and essay prompts from the first study and assembling test 
packages for trial. The team d^sded to make revisions within tlw exi^ng frameworic rather than seek new 
texts or item types. The tests were then administered to students in the 10th to 12th grades in a number of 
volunteo* schools in CaUfomia. New York, and Wisconsin, and the results were analyzed to detenmine score 
profiles and interrelationships. 



Text Selection 

The measures focused on six tents: "The Man by the Fountain** by Gewges Hebbelinck and The 
Use of Force" by William Carlos Williams (fiction): The Biith of the Moon** by an anonymous author and 
-The Iks'* by Lewis Thomas (non-fiction): and ''Forgive My Guilt" by Robert Tristfam Coffin and 
^'Dandelions'* by Deborah Austin (poetry). These texts were selected to provide a range of structures* 
language uses, and degrees of persmal involvement on the part of the writer. They were not intended to 
repre^nt the total range of writers and culmres in the cuniculum* but to maximize differences in text type. 
Three of the texts ("The Man by the Fountain." The Birth of the Moon/ and "Forgive My Guilt") had 



Three of ihe lexis ("The Man by ihc Fountain;* The Binli of the Moon/ and "Forgive My Guilt") had 
been used in a prior study by the Center (Langer, 1989) and il was thought that eventually resulic of the 
two strands oS leseairJi could be confuted 



''The Use of Force" is a nra-person story of a doctor's attempt to examine the throat of a young 
girl and his realization of how violent he has become in assaulting her. The Man by the Fountain"* is a 
third*person story of an old man's brief ^counter with a young runaway whom he finally turns over to the 
police. '^The Iks'" is a reflective essay contrasting an anthropologist's view of a savage tribe with the 
author's own. ''Birth of (he Moon** is a prKess-orient ^ description of the theories .^surrounding the 
separation of the m jon from the Earth. ** Dandelions'" is a humorous poem about a homeowner's battle 
against the dandelions in her lawn that relies on a single extended metaphor to make its point. And 
"Forgive My Guilt'* is a hunter's reflection on the maiming and kilUng of two birds. 



Item Generation 



Measures of Reading 

For the cognitive measures associated with the texts» the questions were limited to a set of 
common topics that could clearly be related to literature education. The following topics were selected to 
provide a range of items across the literary and nonliierary aspects of the domain: content, character 
motivation* over-all structure, and figurative language. Short answer questions were generated for each of 
these topics. 

Measures of Writing 

The extended response items were the subject of some discussion. The research team sought to 
maximize the possibility for the expression of understanding to be helped rather than hind^cd by the test 
situation. The paper, by Gunnar Hansson (19^), suggested that many students wo^ confounded by having 
to read an tmfamiliar text and then compose a formal essay on it. His studies had shown that an 
interm^ary step, such as having the suidents select an illusu^tion or a diagram, draw, or fill out a rating 
form, improved the quality of the oral or written discussion of the text The team therefore created 
alternatives to normal essay questions for four of the six sekxtions. One asked for students to select one or 
more photographs (sec Appendix 2), one askol for a map, one for the compaiative rating of two characters, 
and one for the selection of an organizational diagram. For the other two selections, "The Man by the 
Fountain** and "The Iks," a ncxmal essay question was used. 



Measures of Background Knowledge 

Measures of extratexuial knowledge and of preference (aesthetic, interest and auitudcs) contained 
some general items as well as items spectflc to texts. A measure (tf knowledge of terms was generated from 
a list of terms that were used by suidents in their discussions of the texts in the previous study (Langer, 
1989) together with parallel items for the new texts (e.g. irony). Background knowledge items dealing with 
myth were adapted from ^y National Assessment it'4ns, (NAEP, 1973). Some new items were mlded to 
rei^esent a l^oader selection of myths and Figures. 



Measures of Preference 

The other sets of items that had already been devel(^)ed and validated were the tiansfo* and interest 
items (Purves, 1973) and the censorship items (Dwycr and Summy, 1988). The last were adapted from a 
measure for librarians, ai^ were deemed appropriate since a number of cuiruiula expr^y state tl^ aim of 
having students be readers open to new experiences and tolennt of diverse content in literature. The 
aesthetic judgment items were adapted from the validated mrasure developed by Fiedholm (1974), ba^ on 
the major criteria used for making aesthetic judgments of literature. The only new set of items was one 
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designed to elicit the students* attitudes towards the text, i.e., whether they considered it literary or not. In 
e&ch cose, scaled measures seemed an economical approach to a complex i^ue. 



Th^ questions all written ami rev iewed by an tntmal jury and then sent to an external jury fc^ 
comment and review parallel to the pilot^testing. The complete set of items (without the texts) is included 
in Appendix 2, 

Test Packets 

From this collection of measures, a test packet was create that included in Part 1 a text with its 
shm answer qt^stions* the aesthetic rating, and the background knowledge measures. In Part 2 w«re another 
text followed by tl^ extendi ans\/er measure and the transfer, intoe^ ^ censorship questionnairca. The 
packets were designed so that each student would respond to two different genres in the short answer and 
extended respmse portico of the measure* but all other items would be common. Because of limits in class 
size, the number of paired passages were limited to six rather than ^ full relation. The packets were then put 
into packages by class so that tt^ class woidd respond to the totiJ variety of texts. Each test i^ickage was 
designed to take two class periods (approximately 90 minutes of te^ng), and was designeid to include 
measures lot each of the three subdomains acccxtling to tl^ 5p(x:ification$ given above. 

Sample, Administration, and Scoring 

The tests were disoibuted to a sample of schools selected by the language arts coc^nata* for each 
of ihree slates: California, New York, and Wisconsin. Hie team wrote to each school asking for the number 
of classes and students. In each case the letter was followed by a telephone call. From this response a 
sample was derived as follows (Table 2): 

Table 2 

Sample 

2£lififiis smdsms 

Sent out 25 1554 

Returned 25 1132 

Usable 18 857 



The resulting number of usable packets is disappointing since each of the schools had indicated an exact 
number of classes and students that would participate. Several of the schools, however, reoimed only ok of 
the two packets of test {^pers, so a complete i»t was not available. One school return^ a blank set and 
another a set with no identifying code. Despite these limitations in return, there were suiticient cases to 
derive meaningful results. 

Scoring 

The selection measures were scored right or wrong according to a predetermined key or given the 
value indicated by the student Both the stK)rt answer and the extended reqxNise ttms were scored on a four- 
point scale, with a ''O" for blank answers. For these responses, the scoring was und^rtakra by one rater, 
with a second rater check*$coring every tenth paper. The two raters did fK)t discus dlie papers except to 
agree on certain basic rul^ Agreement, therefore, calculated on a pool of 782 items of which 605 were 
responses that could be given a scale score (i.e. were not blank). The results of the exercise are as follows 
(Table 3): 
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Scoring Agreement on Subsample of Open-ended Iicms 



Items Aptscmeni 1 -point 

Reading 254 61% 33% 

Knowledge 309 76% 23% 

Writing 42 81% 17% 

TOTAL 605 70% 27% 



This result diows that the scoring can be done with a reasonable degree of agreement wiihou* any 
special training, and could be improved with a rater training period. The derived scoring guides and scales 
can be found in Appendix 3. 

III. Results 

Univariate Results 

The Kjores for each of the measures prove to be consistent with the pilot study (Sec Appendix 4. 
Tables 1-14. and the comparable tables in Purvcs. Li and Shirk, 1990). The item statistics for the various 
short-ans^' r «e^tk>ns ^owcd a range of p-valucs within pass^e sets and background measures (Appendix 
4, Tables ,and9). Several items were quite easy and a few were veiy difTiculi, suggesting that these 
measures .^c well within the capacity of the students. There were two measures of knowledge, one on 
cultural information and one on literary terms. Both of these also proved to have a range of difficulty. In the 
culuiral infonnation portion, the most difficult items were those dealing with Faust (8.9% correct). John 
Henry (17.6%), and Oedipus (18.4%); the easiest were The Trojan Horse (70.6%), Noah (67%), and Robin 
Hood (66.9%). In the literary terms segment, the easiest were climax of a story (n-can score 2.47 on a 3- 
point scale) and symbol (2.15); the most difficult were ononiaicpocia (1.14) and sonnet (1.23). The 
complete item statistics for these measures can be found in Appendix 4, Tables 13 and 14. 

Figure 1 shows the relative scores on the six passages, based upon the short answer, the extended 
response and the rating. It is clear that the difficulty of the passage depends in part upon the type of 
response required. If the student were asked short comprehension questions, "The Use of Force" would be an 
easy passage to comprehend, as would "The Binh of the Moon." When one turns to the extended response, 
however, one finds that students provide highly acceptable responses to "Forgive My Guilt" This result 
appears to be an artifaci of the kind of question. Clearly selecting a picture and justifying its use to support 
a mediated interpretation of the text allows more students to perform better than does a "straight forward" 
comprehension measure. This interpretation is not gcneralizable m«ws texts, hotwvcr. for the short answer 
lespoosc to "The Binh of the Moon" produced highw scores than did the mediated response. Given th* size 
of the sample and what we know of the scoring of open-end^ responses in general, it is hard to disentangle 
task effects from student effects and rater effects. We suspect that this inieipretaiion applies to most 
measures of the understanding of literature, for the difficulty of a text appears to depend more upon the 
situation and kinds of questions asked about it, than upon any intrinsic characteristic of the text itself 
(Purves. Li, Shirk, 1990). 
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Figure l: Stoadftrd Scores on Tbree llemsuree By Selection 
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The other observation frwi this figure concerns the results on the poem, "Dandelions." This 
proved the hardest text of alf, regardless of the mode of respc^se; it was also the least liked of the 
selections. Analysis of the responses indicates that ihe vast majority of the students simply did not perceive 
the metaphor in the poem which is signalled only by the title* Even the mediated response task did not help 
the students clarify the metaphor. They generally read the poem as a poem about a battle rather than one 
about i>t blossoming, seed-setting, and death of the common plant. It is clear that this text should 
probably not be used in a large-scale measure. In a separate trial with university undergraduates from the 
Eastern subtirbs, the poem proved equally elusive without exter^ive discussion of various details. 



A summary of the mean scores for all measures is presented in Tabic 4. 
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Table 4 



Summary Statistics 



Ptactke-Readiny M^Ul Std. Dev 

Slory(0.18) 15.32 3.44 

Poem(0-18) 13.92 3.53 

Nonnctjon(0-18) 14.84 4.30 

PrKitice-Writing 

Sioiy(0-3) 1.82 1.03 

Poem(a-3) 1.97 .94 

Nonfiction(0-3) 2.02 1.08 



BaiJcymund Knowledge 

Cultural Information (0-14) 5.69 3.72 

Uterary Terms (0-30) 18.19 8.63 



F)ref««nccs Rating 






Stoiy(0-6) 


4.05 


1.33 


Poem(0-6) 


4.09 


1.22 


NOTficlion(O^) 


3.91 


1.39 


Transfer (040) 


24.21 


5.50 


Interest (0-36) 


18.66 


4.97 


Cens(»stiip (0-%) 


47.48 


10.90 



The summary indicates that the sample as a whole performed l>eIow the midpoint on the cultural 
information items and above it on the liters terms. 

When one turns to the measures of habit and preference* one flnds first that the poem, "Forgive my 
Guilt"* was the most popular and the poem ''Dandelions** the leasK but also that as genres, the stories were 
favored above the nonftction. The most frequently selected criterion for making the judgment was ""The 
emotional impact of the writing.*" It was the most popular choice for the poems and the stories. For the 
nonftction* the most popular was ''The way the writing is organized^ in fact, the second most popular 
criteria foi the nonfiction woe "^The social relevance of the selection** and **The ability of the writer to draw 
me into the experience." The first was not among the most popular two for any of the literary texts. aiKl the 
second was for all but **Dandelions.** It would seem that the structure of the piece plays a greater role in 
students' judgments of nonflction than it does in their judgments of Ut^ture where affective crit^a ccxne 
into play. But we can also see from Appendix Table 6 that students generally choose personal criteria, those 
which we might most readily associate with a lack of training in the forming and defending of aesthetic 
judgments. 

If we combine the fr^uencies for fu^ and for second choice for each of the selections, we find that 
the correlations (using the Spearman Rank-Order Correlation) among the ranks of the criteria chosen shifts 
dramatically: 
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Table 5 



Rank-Order Correlation of Mosl-Favored Criteria 







Guilt 




Moon 


Iks 


Force 


.69 


.60 


.50 


.37 


.28 


Fountain 




.80 


.53 


.41 


.29 


Guilt 






.67 


.56 


.21 


Moon 








.59 


..^2 


Iks 










.28 



From this table, we see that the criteria us&i to judge ''The Iks" differs most greatly from those used (or the 
other selections, even the other nonfiction selections. The strongest similarity is that between '^The Man by 
the Fountain'' and TOTgive my Guilt/ There does not appear to be a strong genre effect on the students' 
criteria; rather it Is a passage effect* We only have a report on one selection from each student, so we cannot 
say whether the students tend to use the same criteria ftom selortion to selection. 



The other measures of habits and pref^nces Indicate that the sample as a whole tend to become 
involved in what they read, to be moderately interested in literature, and to be mixed in their beliefs 
concerning censorship. For the last scale, the mean is 47.48* just under the midpoint, but the standard 
deviation is iKarly 1 1 points, which suggests that the population divides itself sharply on this issue. Such a 
portrait, however, needs to be modified by an examination of the relationships amc^g the various scores 



Relationships and Patterns 

The size of the sample is sufficient to explore the full range of relationships among the various 
subscales. The initial correlation matrix is presented in Table 6. In reading the results, one should take into 
account that the reliability of the writing measure is iow. and that of the reading measure depends to some 
extent on the text read. Nonetheless some tentative relationships emerge* 
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Table 6 

Correlation of Comprehension. Rating. Essay. 
Term. Legend. Transfer. Interest and Censorship 



RAT- 



LEG- TRANS- INTER- 



READ. 


ING 


WRITE 


TERM 


END 


FER 


EST 


CENSOR 


READ 


32* 


.29* 


.47* 


.39* 


.16* 


.20* 


-.03 


RATING 




.14* 


.22* 


.20* 


.11* 


.05 


.05 


WRITE 






.33* 


.30* 


.13* 


.15* 


-.04 


TERM 








.55* 


.16* 


.24* 


-.04 


LEGEND 










.15* 


.26* 


-.01 


TRANSFER 












.48* 


.04 


INTEREST 














,05 



* P< .05 



It is clear that except for the censorshir scal<J. there is a small but ambiguous relationship 
among the measures. The measures within Fabdoniains are more closely related than are the 
subdomains to each other. The independence of the censorship scale suggests that it is an independent 
aspect of literature learning, to the extent tha peop'^ develop beliefs about censorship thai are not 
influenced by not do they influence their comprehension, background knowledge, qualities as readers, (x 
intoests in reading and literature. It may also be the case thai beliefs about censorship are dominated by 
family, home, and social conditions and not affected by the curriculum, despite the claims of various 
interest groups. 

Apait from this independence, we see a strong relationship between terms and legends, the two 
aspects of background knowledge, and betw^n transfer and interest, two qualities of the students' 
preferences as r^dos. Although each of the sub^ales for these two domains remains independent, there 
is a relationship. Students who know their cultural background know their literary terms, and people 
who are deeply engaged readers also tend to be readers of habit. 

One surprising result is that "comprehension" and "essay." the practices of reading and of 
writing extended responses, tend to be independent of each other, more so than one might expect. 
Whether this low correlation reflects a task efl^ect or a text effect is unclear and certainly needs further 
exploration. It is also appanni that "good" readers in the sense of being ^le to answer comprehensior < 
questions or in the sense oS being able to write extended prose are not nece^arily good refers in being 
habitually involved in what they read in being interested in literature. One may question whether the 
critical reader is the same as the committed readst. It is certainty not the case that the good writer likes 
the selection that is being read; the good "critical reader." however, may. Again the aesthetic judgment 
is related to, but not clearly predictive of the quality of understanding or the habi: of being involved in 
what one re^. 
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When one examines these relaiionships through the technique of muhiple regression, to 
examine the relationships across subdomains. one finds a set of results that confirms the impression of 
the correlation. The first regression analysis used the comprehension score as a dependent variable 
(Table 6): the second used the Qssay score as a dependent variable (Table 7). 



Table 7 

Stepwise Regression Using Reading as a I>ependent Variable 



StSC Yaoatzk ConSlalUm Beta Weight* Cumulative Percent 

of Explained 

1 Ternis .46 .46 .21 

2 Rating .28 .19 .24 

3 Legend .38 .17 .26 

4 Writing .29 .12 .28 

5 Transfer .17 .06 .28 



* All beta weights are significant at the .03 level or better. 



Two points are of interest when one considers the results of this analysis. The first is the 
considerable importance of knowledge of terms to high performance on the measures of practice. The other 
measures contribute to achievement, but do not add very much to the amount of variation that can be 
explained. It seems clear that the "able reader" of literaiy texts as demonstrated by comprehension questions 
is the person who has learned the . metalanguage of literature and its criticism. The second point is the 
relatively small amount of explained variance in tolo. The aspects of literature learning are relatively 
independent of each other. 



Table 8 

Stepwise Regression Using Writing as the Dependent Variable 



^ Yadabk Correlatifm Beta Weight* Cumulative Percent 

of Explained 
Variance 

1 Terms .33 .33 .11 

2 Reading .29 .17 .13 

3 Legend .30 .14 .15 



* Ail beta weights are significant at the .0005 level or better. 



When one uses the essay score as the dependent variable, knowledge of terminology again plays an 
important predictive role, but much less of the variance is explained. This finding results in great part from 
the lesser variance of the compositions, but at the same time indicat&s that the more adept writer tends to 
have a store of backgroui^ knowledge. 

The correlations and regressions, however, generally indicate the lack of interdependence of the 
measures and suggest that logically related as the subdomains may be. in practice student performance does 
not present a unifonn picnire. 
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Studeni Comments on the Tests 



The students were also asked to comment on the tefts after they had taken them* and a large 
number did indeed voice their ccHnments. The most oft-stated comments by students concern the Attitude 
Measures and their responses to this measure seem to fall equally into either favoring such questions or 
finding them **only cvintcMi*" questions and irrelevant if not impertinent. They are either pleased to be asked 
about their ideas about literature, and their responses to it, their literary habits and tastes and views on 
censorship CIt*s good to fmd out how kids think and feel about life") or they are cynical about these 
questions CI did ncM like the seccHid pan as much as the first. It seemed like propaganda to me with that 
quiz on censorship**). Even among those who were thankful for being asked qi^tions of opinion, there is 
expressed a distrust of being considered genuinely or seriously. Tm glad their (sic) is a survey in here, it at 
least makes kids feel like their opinions count for something, even if tl^y don't.*" was surprised at the 
questions about censorship. It may or may not CHily be a survey. In the end it won*t matter one way or 
another. Older needle think of a(k)lescents as children. They dcxi*t take their concerns (^vice) seriously but 
thank you^anv ay, for trying.*" 

Some exfTCssed that they would have liked an opportunity to exfrcss the reasons for their opinions 
and found the method of answering (frequency scales, and likert scal^) objectionable because they do not 
provide an opp(»iunity to do this. And many used the student response form to elaborate on their opinicHis 
concerning censorship particularly. 

Many students object to the test as a waste of time or not relevant to them in any way because it 
is not practice for the Regents or any of the other tests they must take. And a test for which no credit is 
given is a '^waste of my time.** But pei1iai» the most perceptive and sensitive responses are those that see 
the test as perhaps a harbinger of improvement. '*From what my teacher has told me and from taking this 
test, I feel this test will be very beneficial if things are done to change some t^d parts of the systems, (in 
libraries, schools, regents,etc..).'' ""I feel that these tests are an excellent idea because you learn about the 
students icteas which aren^t express^ through normal high school work. More of these tests should be given 
to students everywhere so maybe the school system would change to teach the necessary things that [a] 
person needs to know in life." 

Although there are a few students who perceive the test as an alternative to miditional testing, an 
attempt to broken the field of literature smdies in the leabri of the clasmx)m and the testing scene, for the 
most pan this is sten as just another task to perform. Perhaps the students, like the teachers, should be 
better informed as to the researcher's design and agenda. It is disheaitening to consider that for a maprity of 
the students this research is just anoth^ task in a fragmented day to be viewed apathetically as purposeless. 
Together with the results on some of the measures, these ccnnments by the students can raise questions 
about the nature and effect of the literature curriculum. 



IV« Conclusion 

The exercise in creating a domain-referenced evaluation of literature learning at the secondary 
school level brings with it wme conclusions about testing. Concerning the domain of literature, the test 
results indicate that the three subdcmains of knowledge, practice, and habits or preferences are more distinct 
than they are related. They tend to interpenetrate each other to a lesser extent than curriculum planners and 
theorists might predict. Knowledge affects practice but is not a substitute (or it; the same can be said for 
interests and qiialities of reading. Within the field of practice, there seems to be a distinction between 
demonstrating one's understanding of a text wd articulating a sustained response to a text. Reading and 
answering someone ek^'s questions and writing out one*$ own understanding of a text are indeed related, but 
hardly equivalent 

The tests that we have devised are imperfect measures as all tests are. The measures may be seen as 
inconsistent in that they ask the students at one point to match the examiner's uiKlerstanding of the text and 
at others to encourage an openness of response possibilities. This inconsistency can t>e construed as a vice 
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or a vinue, of course, but we would argue that it is an inconsistency within the curriculum that presently 
exists (Apptebee. 1990). The students' comments suggest that they perceive that inconsistency. 

In their totality, however, these measures or a package of measures similar to them, can help 
enable a school, a district, or a state to get some picture of what students of literature look like. Results 
fipom such an assessment package helps fom a portrait of the typical product of our schools. Although 
they could be used for student evaluation, we believe that their best use is to evaluate programs, to show 
the relationships between the intended literature curriculum and the achieved curriculum in a given school or 
classroom. 
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A Note on Iten ClassificatioR 



One of the most common ways of classifying u st items in iit^ture is by the content of the item, whether 
and in what way it refers to the text. Another CMtmon way has been in terms of the format, whether short 
answ^ or multiple choice. A third has been acc:HYling to ite Taxonomy of EdtK^tioial Objectives, a project 
of the 1950*$ that attempted to ctefine items in terms of the cognitive skills they called for (Bloom, et al.« 
19S4). This last way has proved somewhat controversial in that the project discriminated between the 
cognitive and aft'ective domains and attempted to establish a mental hienirchy. Assessment projects at the 
Center for the Learniiig and Teaching of Literature have been examining literature tests ' vm 
tht perspective of content and cognitive skills (Brody, DeMilo* and Purves. I9g9)« with success on the first 
peisp^tive and less w the second. 

It seems that te» questions might be^ be examii^ in terms of a combinadon of format and ocmand upon 
the ^udent Such an approach parallels the work on composition tasks performed by Vflh^passi ( 1988) who 
saw them as an interaction of cognitive demand and discourse function. With r^p^t to taisks rented to the 
reading of literature, this combinatiw would en^Ie one to ^ that the task confronting the student, for the 
most part, has to do with the degree to which the student must ^pply new infoimation rather than use 
information that is present. This degree is related to the degree to which the student must assimilate the 
particular text and come to conclusions about it. which conclusions must then be articulated. Such an 
approach is related to that set forth in the work of Hansson (1990). It is clear that in an ^say question, for 
example, a student mu. nniy information in order to articulate a response, drawing upon long-term 
memory and« to some exteni, ^ erating new infonnation and new structure. In a multiple-choice qi^stion. 
on the other hand, the student must kx)k at the inform^ion presented and select the best match between 
stem and options. In a literature t^t the student may also need to have recourse to the text, but the student 
need not gen^ale an articulated interpretation of the text in ontor to g^ the '*right'* answer. From these two 
extremes, we may peiliaps develop an item classification system that will be of use to those constructing 
and analyzing tests in literature which involve knowl^ge. practice and preference. It may be too that there 
are implications for the cuiriculum. 

I. Items where all infccmation is present or avaibbie. 

A. Matching 

i. Picture-text 

ii. Text-text 

iii. Interpretation-'text 

B. Sorting 

i. Pictures 

ii. Words^hrases/sentences 

iii. Continuous text 

C. Reorganizing 

i. Pictures 

ii. Wonki^hrases/sentences 

iii. Continuous text 

II Items where the majority of information is present or available, the rest is presumed to be in the long- 
term memory and is to be applied to the present situation (cf . Broudy 19S2). 

A. Supplying 

i. Pictures 

ti. Words/ji^hrases/sentenc^ 
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iii* Continuous tcxi 



UL Items where the information was prci»cnt in a previously read text and is presumed to be in the long- 
terni memory. The pnmary task is to make a report of what is known* usually within a prescribed 
form. In Broudy's temis. the use of learning is primarily replicattve. 

A. Recalling 

i, WordVphrascs/scntcnccs 

ii. Continuous text 

B. Par^hiasing/Summarizing 

i. Wcx'ds^hrascs/scntcnces 

ii. Continuous text 

IV. Items where the uiformation is not presumed to have been directly [ resent in the long-term memory 
but in which the student is required to make an interpretive or associrdve use of prior learning (Broudy, 
1982). 

A. Constructing/generating 

i. Words/phrases/scntcnccs 

ii. Continuous u:xt 

iii. Interpretation 

iv. Synthesis/judgment 
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Appeodix 2 

Sample Measures of the Domain of Literature Learning 



THE MAN BY THE FOUNTAIN 

For each of the following questions write out your best answer in the spxc that rollows. 

1. How do the descriptions of the fountain differ from each other? 

2. Why does John Deweck wish ""sor * tittle boy would turn up?" 

3. How does the boy respond U) John when John initiates a conversation with him? 

4. How do the boy's feelings for John change as the story progresses? 

5. At the end of the stoiy why does John Deweck i^ver again return lo the fountain? 

6. What do the different descriptions of the fountain contribute to the development of the relationship 
between Jolvi Deweck and tte boy? 

THE USE OF FORCE 

L Describe how Mathilda responds to the doctor at the beginning of the story. 

2. How does the doctor respond to Mathilda at the beginning of the story? 

3. Why does Mathilda claw for the doctor's eyes? (lines 47-50) 

4. How do you think tte doctor's atlinjde toward Mathilda changes as he proceeds with the examination? 

5. How would you describe the doctor s responses to the parents? 

6. How does the doctor's perception of himself change from the beginning to the end of the story? 
BIRTH OF THE MOON 

L Why is the Impactor Theory a favorite of astronomers? 

2. What similarities and differences between the earth and the moon arc revealed by the moon rocks* 
Trngerprint**? 

3. Compare the way the astroiomeis quoted differ in altitude toward the Imp»:tor Theory. 

4. What do the words **imi»ctor\ "whark**. and '^^lash" have in common? 

5. Describe the way the Impactor Theory is presented and explained in this essay. 

6. How is the Impactor Theory prt^nted differently in the first and last paragraphs of this essay? 
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THE IKS 



1. Whai happen^ to the Iks that made them ''celcbriiics"? 

2. How do the narrator's ideas about the Iks differ from the anthropologist s ideas about them? 

3. What assumptions must we agr« with to accept the anthropologist s theory about the Iks? 

4. Describe the organization of this essay* 

5. How V ould you describe the author's attitude towards the anthropologist? 

6. What do the comparisons the author makes between the Iks and social systems contribute to the way he 
presents his view of the Ocs? 

DANDEUONS 

1. Where do you think this battleField is? 

2. Is there a victor in this battle, who oi what do you think it is? 
3- What conflicts are refvesented by the battle? 

4, What does each ''pow" represent? 

5- Describe the way the poet uses the progression of time in this poem. 
6. What senses are c«nbined in the images in the poemi 

FORGIVE MY GUILT 

1. From whom is the poem asking forgiveness? 

2. How are the birds' songs described? 

3. What keeps the memory alive? 

4. Why is the speaker of this poem asking to be forgiven? 

5. What connections are there between the two stanzas of tf^ poem? 

6. What do you think the comparisons the poet makes about the wounded birds contribute to the poem? 
RATING 

L We would like to know how you personally compare this story to other stories you have read. If you 
think it is one of the best stories you have read, rale it ^3. If you think it is one of the worst you have 
read, rate it -3* 

Please draw a circle around the number that is closest to your mting: 

One of the best Good Fairly Good Fairly Poor Poor One of the worst 

+3 +2 ^^I -I -2 -3 



ERLC 



19 



2- 



Here is a list of qualities that some people have said fonn the basis of their judgement of a piece of writing. 
Look over the list and choose two qualities ihat most influenced your judgement of this piece of writing. 
Write the letters of your selections in the space provided lielow, 

A. The truth-to-life of the writing. 

B. The way the writing is organized, 

C. The emotional impact of the writing. 

D. The moral lesson in the writing. 

E. The language and style of the writer, 

F. The origirtality of the writer. 

G. The social relevance of the selection. 

H. The ability of the writer to draw me into the experience. 

I The way the writer used various metaphors, images, and symbols lo keep me thinking. 

J. The personal relevance of the selection. 

Qualities selected: 1. 

'J 



BACKGROUND KNOWLEDGE 

There follow fourteen story synopses or starters. Each is based on an old legend. Below are fifteen names 
of legends. In the blank space after each story write the letter corresponding to the legend upon which you 
think the story is based* 



A. 


OEDIPUS 


B. 


THE FALL OF ADAM 


C. 


HAMLET 


D. 


FAUST 


E. 


JOB 


F. 


ROBIN HOOD 


G. 


ARTHUR 


H. 


NOAH 



I. THE FALL OF ICARUS 
J. TOM SAWYER 
K, CAIN AND ABEL 
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L. THE TROJAN HORSE 



M. JO>'N HENRY 

N. THE ODYSSEY 

0. ROMEO AND JULIET 



1. Old Petersen had had a good hfe. a good farm--the richest in the country, a happy family, a nice tidy 
income. Even so. he wasn't proud, just thankful. Then the locusts came and ate his wheat, someone 
poisoned his wells, and to cap it off. his children died of dipiheria. What happeneU. Petersen wondered, 
what had he done? It was enoijgh to make a man lose his faith. 

Answer 



2. How would they ever recapture the fort from the Apaches? They had tried everything - raids, mass 
atUKk. cutting off the water-but nothing had worked. Then Sei^geant Gottschalk had an idea. He put a false 
floor in a Conestoga wagon, hid ten of his best men under it. loaded the top with supplies, and drove right 
into the town. "We're letting you have the fort." he told the chief, who watched the troop pull out. That 
night, however, the ten men sneaked out and opened the gates and the Indians were surprised to see the troop 
returning. 

Answer 



3, Orville came into the room and told his mother. "Now that I've finished high school. I'm going on to 
learn everything about everything: science, history, music - everything. I don't care how 1 get this 
knowledge: just so 1 get it. Then IH know everything, I'll have the power I want." 

Answer 



4. Raul was angry: the drug lords controll«J the streets an^J the law and the citizens were terrified. Talking 
to two of his friends, he said. "If the crooks have the law on their side, we ll be the outlaws. Let's take 
their profits and use the money to help the addicts' families." 

Answer 
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S. Charlie whistled as he thought about it. Everything had gone right for him after he'd left ReadviUe, on 
the other side of the divide* just because he'd been warned by the gypsy he was in for it. He'd come into 
Melrose and just happen^ to stop the gang that had been terrorizing the town. So they made him shcrifr 
He'd married the mayor's widow, a lovely w^man even if she was olckr. '*Yes/' he thought, '1 guess that 
old gypsy was wrong, Tve sure found happiness ncM trouble/ 

Answer 



6. Harry's weekend was ruined. How was he going to go on the overnight if he had to clean up the yard? 
He got out the trash bags and rake and moved to one corner. '^Watcha doin. Working?" called Danny. 
"Nope, not work^ said Harry. '*Building up the upper arms for the swim meet." •'Let me try/ said 
Danny. "Nope, too hard." Harry responded "Please let me do two turns." Danny pleaded. "Well. OK." 
said Harry. "What!! you give me?** There was now a gleam of hope in Harry's eyes. 

Answer 



7. Ernie walked into the dance not sure what was going to happen. His crowd had fought with the group 
that was giving the party but he was in a mood; his girl had just left him. He was spoiling for something. 
He looked across the room which was smoky and dark. He saw her and knew this was it. He walked over. 
"Hi, Who are you?" She looked up and he could see the light in her eyes was like his. "I'm Linda." 



Answer. 



8. Sam had had such high hopes, such trust that all was well here in the fruitful valley. To him, 
everything summed good. Then, somehow , something went wrong, something spoiled it. It all started 
when that fancy salesman carne in and got Sam's wife to buy his encyclopedia. 

Answer 



9, Now that he had proved that only he could remove the lazer-sword from the asteroid how was Zarkon to 
keep the galaxy at peace? There had been centuries of planetary wars and now the representative lords had 
agreed to end it and h^ named Zaifcon empercs- (rf the Federation. The android macte his first announcement. 
"Choose nine of the best warriors from exh of the eleven planets. Wartle each to one of the chair-screens 
around the hall. I will Icai them to end war and rid the galaxy of injustice, ra* we will die in that effort." 

Answer 



10, He hadn t been sure before, but now he was. His !itile brother, Sid, was the one who was going to be 
the success; he was going to get everything. It wasn't that Sid was better or brighter or anything, just 
luckier. And this made Sam mad, so he wanted to kill his little brother. And one day he did. 



Answer 



1 L Sean didn't know what to do. His father had died mysteriously - was it cancer? Now his mother was 
about to marry UiKle Ed. It was so sudden. Sean was suspicious. But he needed proof. 

Answer 



12. Elaine had been waiting - how long was it? Eddy had been gone twenty years - first in Vietnam and 
then louring through the South f^ific. Each time hed been about to come home something came up. 
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Meanwhile, lots of guys had been trying to get her to cfeclare herself a widow and marry ihem. She'd tried 
eveiy trick to stall them. 5he*d heard Eddy was on his way home. 



Answer 

13. The planet was doomed. Nick knew it but he also knew others didn't care. So he decided lo try to save 
as many people and animals as he could and colonize the next galaxy. Everyone laughed but he went ahead. 

Answer 

14. Rose was a master calculator. She could figure out a balance sheet in less than an hour given the raw 
numbers. The company thought she was indispensable. Then the new director brought in a computer 
expert He challenged Rose to see who could do the books fastest and most accurately. It was a close race 
but she won - at what cosi? 

Answer 

In a brief paragraph for each, write a definition of the following terms: 



1. 


Metaphor 


/% 


Symbol 


3. 


Onomatopoeia 


4. 


Pei^onification 


5. 


Protagonist 


6, 


Climax of a story 


7. 


First-person narrative 


8. 


Irony 


9. 


Sonnet 



10. Rising action 
EXTENDED RESPONSES 
FORGIVE MY GUILT 

Look at the pictures on the next pag^. Select the one or two that you think best illustrates the mood(s) 
expressed in "Forgive My Guilf. Write an essay in which you explain your choice. 



ERIC 



pc 

23 



DANDEUONS 



Look again at the title and then at the poem. Draw a sketch map of the scene and wriie a journal of the 
''batt'e'* frntv Day 1 to Day 3 in which you plot the changes on the map. You may make 3 sketch maps if 
you prefer. 



THE MAN BY THE FOUNTAIN 

Write a composition of two or three paragraphs in which you compare the different images related lo the 
fountain and their effect upon your understanding of the story. "The Man By the Fountain." Your 
composition will be judged on the quality of your answer to the question as well as on its organization, 
style, and use of the language. 



THE USE OF FORCE 

Please rate the doctor and Mathilda on each of the following using 1 to indicate low m this scale and 5 to 
indicate high. 

Doctor Mathilda 
Beginning End Beginni.ig End 



Good 



Strong 
Active 



Proper 
ImmOTal 



Powerless 

Using your ratings as a basis for a composition, describe the changes in the relationship between these two 
characters in **The Use of Force." 



THE IKS 

Write a composition of two or three paragraphs in which you contrast the views of the author and the 
anthropologist concerning the Iks. Your composition will be judged on the quality of your answer to the 
qu^tion as well as on its organization, style and use of the language. 
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BIRTH OF THE MOON 



A. Uncenainiy Certainty 

Fact 

B. Theory Opinion 

Question 

C. Myth Reality 

D. Literal Metaphor 
Sciemre Art 

Here are four diagrams that might be used to de^rribe the structure of "The Birth of the Moon. " Do you 
think any of them fits? Which one? If you don't like any of them draw one of your own. Write an essay 
in which you show why the diagram you selected or drew describes the essay best. 
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ATTITUDE MEASURES 



For each of the questions that follow, select the one answer thai best matches your experience and beliefs. 
Circle the letter that ccHresponds to your choice. 

1. Have you done something you would not ordinarily have done because you read about it in a story, 
poem, or play? (For example, when you were younger, have you dressed up as a pirate because you read a 
st(H7 ^ut pirates?) 

a. Often 

b. Occasionally 

c. Once or twice 

d. Never 

2. While you were reading a book have you thought of yoyrself as one of the people in it? 

a* Often 

b. Occasionally 

c. Once or twice 

d. Never 

3. Have you compared a person you met in real life with people you have read about? (For instance, have 
you ever called a stnMig person Samson?) 

a. Often 

b. Occasionally 

c. Once or twice 
d Never 

4. Have you been in a situation and asked yourself what some person in a story you read would have done 
in that situation? 

a. Often 

b* Occasionally 

c. Once or twice 

d. Never 

5. When you read a novel or a story, do you imagine that what is happening in the story takes place in 
some town or city that you have seen? 

a. Never 

b. Once or twice 

c. Occasionally 

d. Often 

6. Have you done something or gone somewhere, felt that this has happened before, and then realized that 
in fact it h^pened in a book you read? 

a. Never 

b. Once or twice 
c* Occasionally 
d. Often 

7. When you read a story, how often do you imagine that the people in the story look like people you 
know? 

a. Often 

b. Occasionally 
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c* Seldom 
(L Never 



8. When you meet a new person, how often do you compare the person to someone you saw in a movie? 

a. Often 

b. CX'casionally 
c- Seldom 

d. Never 

9. How often do you think thai the people you are reading about in a story are real people and not simply 
people in a story? 

a. Never 

b. Once or twice 

c. Occasionally 

d. Often 

10. When you read a story or a play* do you iry to remember something that happened to you that is like 
what you are reading about? Do you say to yourself. **Something like this happened to me once?" 

a. Never 

b. Once or twice 

c. Occasionally 

d. Often 

1 1. How many books have you read for ycHir own pteasure in the past year? 

a. None 

b. Fewer than 5 

c. 5 to 10 

d. More than 10 

12. During the past year, how many plays have you read for your own pleasure? 

a. None 

b. One or two 

c. 3 to 5 

d. More than 5 

13. During the past year, how many novels have you read for your own pleasure? 

a. None 

b- One or two 

c, 3 to 5 

d. More than S 

14. During the past year, how many biographies have you read for your own pleasure? 

a* None 

K One or two 

c. 3 to 5 

d. Moie than 5 
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15. When you choose a story or novel to read, which one of the following i.s most likely lo be the reason 
for your choice? 

a. Friends or parents lecomnfiend it 

b. I have read other books by the same author 

c. The title attracts me 

d. I just choose any 

16. How often do you re-read novels, stories or plays? 

a. Never 

b. (^e or twice 

c. Occasionally 

d. T^iequently 

17. Have you ever gone to a movie because you read the story in a book? 

a. Often 

b. Occasionally 

c. Once or twice 

d. Never 

18. Have you ever read a book because you saw the story in a movie? 

a. Never 

b Once or twice 

c. Occasionally 

d. Often 

19. Have you ever read a book because you saw the story on television or heard the story on the radio? 

a. Often 

b. Occasionally 

c. Once or twice 

d. Never 

20. After you have seen a play or movie, would you want to read a criticism of the work? 

a. Often 

b. OccasicMially 

c. Once or twice 
d Never 
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BELIEFS ABOUT LITERATURE 

Indicate iho extent to whicti 
you agree with each of the 
following statements peopte 
have made ^XHit Censorship. 
Circle the number that indicates 
the extent of your agreement. 

Agree Strongly Agree Disagree Disagree Strongly 

4 3 2 1 



1. Librarians in elementary schools 
should remove hooks from cir* 
culation that they believe would be 
harmful to children if read by them. 



2. Clergy (priests, ministers, rabbis, 
etc.) should not have the right to re- 
move books from libraries even if 
th(^ books are offensive to the 
people in their places of worship. 

3. Bodes by Karl Marx should be 
banned from libraries because they 
glorify communism. 



4. Books by convicted criminals 
serving time in prisons should not 
be banned from libraries even if 
the criminals will get rich from the 
sale of the books once they are 
released from prison. 

5. Pornographic m^erials should be 
banned from all libmries and bookstores 4 3 2 1 

6. Teachers and school ^Jministrators 
should not be allowed to take books 
from students even if they believe 
the books contain infomiation harm- 
ful to the students. 4 3 2 1 

7. The production of pornographic 
books should be a serious cri&ne and 
the authors and puUishers should 

be imprisoned. 4 3 2 1 



4 3 2 1 

4 3 2 1 
4 3 2 1 
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8. Faulty should not be allowed to 
prevent students &om publishing 
articles in student newspapers. 

9. Censorship boards made up of 
outstanding community members 
^uld have the power to ban books 
which they believe have dangerous 
ccHitent. 

10. Public and school libraries should 
not be permitted to have books on 
sex education available for reading 

by people under 16 years of age. 

11. Newf *jp^ should repOTt the 
anicles ot groui^ wanting to over- 
throw tt^ government of the United 
States. This is true even if the 
activities might be appealing to 
people not now involved with the 
groups* 

12. A newspaper should not report 
last words of a convict about to be 
electrocuted if the convict said he was 
proud of the murcter he committed 
and was glad he did it and that he 
thought of himself as a hero. 

13. Advertisements for membership 
in the Ku Klux Klan should be per- 
mitted in high school and college 
newspapers. 

14* The United States government 
should not be allowed to pass laws 
controlling advertising in chiWren's 
newspapers. 

15* Public and school libraries should 
not have books about drug use that 
can be checks out by chiktren under 
16 years of age. 

16. Young people are telped in 
learning what is right and what is 
wrong when books with un-Christian 
ideas are kept from them. 

17. Taxpayers pay for the books that 
libraries and schools purchase; theie- 
fbre« taxpayers should have the right 
to determine which books are pur- 
chased and used by students. 



18. Cotins should not have the 
right to rule on whether or not books 
should be banned from the com- 
munity. 

19. Adults should not prevent 
teenagers from reading materials 
writtoi by pec^le who glorify 
lifestyles that ate very disturbing 
to the adults. 



20. A psfftni or group of parents 
should have the right to remove a 
book £rom a public lit»raiy or public 
school cuniculum if they find it 
ofTer^ive. 

21. hooks that describe bow to 
commit crimes or how to destroy 
property should be banned. 

22. Books which say that the 
United States was wrong in its in- 
volvement in the Vietnam War 
should be banned becai^ these 
books could make people feel angry 
toward their countiy. 

23. A book should mx be banned 
even if it says th^ the ancient Eskimo 
practice of sending old. helpless, people 
off to die by themselves in the snow 

is a good idea. 

24. LitH^rians should be allowed to 
obtain every type of reading material 
because lil^aries are merely storage 
areas for books and should not limit 
what they hold. 



Appendix 3 
Scoring Guidelines and Scale 



Scoring Criteria 

General Criteria for Scores 

For the stories and poems the differences between a 3 (excellent) answer and a 2 (adequate) answer is 
based on the analytic interpretive developnoent of the former versus a literal summary or response for the 
latter* A score of I is fOT an inappropriate - an unexplained, not obvious interpretation * or plain 
inconeco^. 

Tte criteria for the distinction between a 3 and a 2 score for the nonfiction essays "Birth of the Mom'" 
and The Iks.'' are correctness, cteiails. and development A score of 1 is for an incorrect answer. 

Usually any inteipretation, or speculation, or attempt at analysis that isn't unreasonable or unexplained 
is scored 3. 



The Man by the Fountain" 

1 . 3 if the answer describes the suggestive, ^venturous quality, the carefree jubilance of the fountain in 
the beginning of the story versus its violence and suggestion of doom later on. The difference between and 
3 and & 2 is the quality of expression, the language the student uses to convey the comparison. 

2. 3 if the answer accounts for Deweck's dislike of other grownups or his attraction to children for their 
freedom and innocence or suggests that he identifies with youth as a way to reject age and the fact of death. 
A scat of 2 if the answer conveys that he wanted someone to talk to. 

3. 3 if the answer tells of the boy's initial reluctance and hesitation with any detail and elaboration. A 
score of 2 if the answer is merely that he was first reluctant to talk to John. 

4. 3 if the answer includes 3 stages from distrust to trust to despising Deweck for betraying him. 2 if the 
answer does not convey the three stages of the relationship but says it moves from trust to disuust, or 
conveys the boy's feelings of betrayal by John. 

5. 3 if the answer includes some analysis or explanation for Deweck's never again going to the fountain, 
or if there is some connection between outcome of the relationship with the boy attempted. A score of 2 
reflects the way in which the aisw^ is expressed. 

6. 3 if the writer anempts to make a parallel between attinides represented by ^he fountain and Deweck. or 
if descriptions of the fountain seen as harbinger of events to come. A 2 reflects the way the answer is 
expressed. 



Hie Use of Force" 

1. 3 if the answer describes the hostility the doctor immediately perceives, she "was fairly eating me up 
with her coki shady eyes^** and elaborates in any way on this or offers any commentary. The score is 2 if 
the answer perceives the hostility in the child's stare. 
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2* 3 if the answer suggests his rather unusual description, "usually attractive and strong as a heifer," or 
anempts any analysis of his reactions. A score of 2 if the answer doesn't account for the doctor's physical 
attraction - most students did not ^'notice" this. 

3. 3 if the answer accounts for self-proiection or to keep her secret from him. A score of 2 if the answer 
is too lediKrtive. 

4. 3 if the writer discusses his anger and overwhelming impulse to overpower her, to achieve his intention 
at all costs, answer must account for his loss of self-control. 2 score if the answer doesn't account for the 
conscious loss of control on the (kx:tor's pan. 

5. 3 if the answo* points to his contempt for them, his loss of reason and obsession in overpowering their 
daughter. Score of 2 if the answer doesn't account for tte doctor s feelings of contempt, rather than mere 
annoyance with them. 

6. 3 if the answer accounts for his recognition of his less of control and obsessive unprofessional 
behavior. Scc^ 2 if the (kxtor^s self-awareness isn't accounted fur. 



"Dandelions" 

1 . 3 if the answer is b^kyrxds. A score of 2 if the answer attempts some kind of inlerprelalion. A score 
of I for a cxie word, or unexplained or undeveloped answer. 

2. 3 if the answer elaborates in any reasonable interesting way on a yes or no answer. A score of 2 is for 
a not well-develc^)ed answer and a score of 1 for a one word yes or no answer without any explanation. 

3. A score of 3 cr 2 for any development concerning man versus nature depending on the development of 
the answer and the way in which it is expressed. A score of 1 for an inappropriate or incorrect answer. 

4. 3 for any answer that mentions dandelions. A score of 2 for an answer that only accounts for f .ifire. 

5. 3 for an answer that anempts any explanation of ambiguity, and accounts for 3 days, yesterday, today, 
and tomorrow. If the answer merely accounts for three days, the score is 2. and a score of 1 for an 
inai^ropiate unexplained answer an incorrect answer. 

6. 3 for any development of hearing and sight. A score of 2 for an answer that just lists the senses to 
which the images in the poem refer. 



''Forgive My Guilf 

1 . 3 if the answer accounts fw ''all that is wild, airy and beautiful." nature, metaphysical forces, etc. A 
score of 2 if the answer only refers to ulnls. 

2, 3 if the answer accounts for figurative language. "Flutes of sorrow'*, "Songs of sorrow/' A score of 2 
if the musical quality of the image isn't accounted for in the answer. 

3 . 3 if the writer accounts {(x ''sounds of sorrow" and to references to suffering in other contexts, A score 
of 2 if answer mentions guilt but doesn't elaborate or develop. 

4, 3 if the answer accounts for the self conscious realization of "sin", or narrator's sureness that he has 
done something wrong in violation of the natural life of the birds. A score of 2 is an undeveloped, 
unetaborated carect answer. 
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5. 3. If the answer accounts for the first stanza describing actions and second feelings and if the answer 
elalxMates on the relationship of actions lo consequences, accounts for cause and effect, etc. The score is 2 
if the answer doe^*t elaborate cm how the stanzas are connected. 

6. 3 is any interpfeiive answer about the birds or the feelings of the boy for his *sin A score of 2 is for 
an answer that mentions in general teims the qualities of symbolism imagery in the poem. 



"The Iks'* 

L A score of 3 if the answer is inclusive: land taken from them* way of life changed, whiten about in 
unfavorable light by anthropologist. A score of 2 for an answer that may not be totally inclusive, but that 
reasonably explains the situations. 

2. A score of 3 or 2 depending on how the answer is developed but it must include the difference of 
opinion and attitude between the anthropologist and tte nanrator. 

3. 3 if the answer is inclusive, ''man is fundamentally a bad lot,*" ''out for himself. ' etc.. "leanted habits/' 
etc. A score of 2 for an answer that suggests the ccsrect answer, although it may ncH be inclusive or exact. 

4. 3 if the answer names or describes organization conecily* anthro{K)logist*s analysis an author's 
refutation, argument, counter-argument, with specific references to the text itself. A score of 2 for a 
generally descriptive answer. 

5. 3 if the answer displays author's negative evaluation of this anthropologist ard says that we must 
regard the anthropologist's assertions skeptically as unprofessional, narrow views. A score of 2 if the 
answer is suggestive of the ccxrect answer but not developed or elabwated 

6. 3 if the answer shows how they are ^cn as representative of modem man and can be viewed 
sympathetically and as an example to heed. A score of 2 for an answer that captures the narrator s 
interpretation of the Iks' situation. 



"Birth of the Moon'' 

1. 3 if it says something about 'unusual chemistry" of the moon. A score of 2 if the answer is correct 
but not as specific as a 3. A score of 1 for an incomect or too vague answer. 

2. 3 for details, specifics: isotopes, iron, magnesium, volatiles. etc.. accounts for similarities and 
differences. 2 for a correct but basically gei^ral answer. 

3. 3 if answer accounts for the acknowledgement oi hypothetical nature of the last quote versus the 
unqualiHed acceptance and sureness of the others. The score is 2 if the answer is correct but not as well 
developed or elaborated as a 3. 

4. 3 if the answer defines them as synonyms in relation to the actions descrit)ed in the essay. Score is 2 if 
the answer is correct but not as sp^ific or well-defuied. 

5. 3 for description of the organization of the essay from unqualified description to exposition, etc.. any 
anempt to name different parts of the essay or explain it with particular reference to the text The score is 2 
if it describes the structure in general terms or just summarizes the text. 

6. 3 if the answer contrasts style of the {^graphs, narrative description versus interpretive commentary, 
or sureness versus hypothetical, story versus scientific view. The score is 2 for a correct but not well 
devel(^)ed or explained answer 
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Score Samples 

"Man by the Fountain" 

QJ. How do the descriptions of the fountain differ from each other*? 

The fountain at first symlwli/.ed good things. For instance people playing in the park. Laicr it 
symbolized bad events about to occur, like the fountain turning violent right before the boy came. 

One description is pleasant and good to lock forward to, the other is evil coming. 

One describes a nice fountain, while the other is a description of a mean fountain. 
It changes as the mood does in the story. 

First it is described as an inanimate object sending water toward the clouds. Then it is relayed as a 
mass of dragons and gargials. 

One is large and grand, ihe other is small 
Q.2. Why does John wish "some little boy would turn up?" 

He wishes to become young again and this is as close to youth as he can get. 

He wants a little boy to show up because he enjoys talking with youth because they do not yet know 
or care about the b^ things in life. He wants to talk to someone with something good to say. 

He needed to talk to someone. He was very lonely. 
To have an intellectual talk (not gossip) with someone. 

He wished some little boy would turn up so he could do his job and send him back to school. 
He is bored. 
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Q,3. How does the boy respond to John when John initiates a conversation with him? 
^ He r^ponds very defensively, but winds up uusting the old man. 

m The boy is not sure he should inisl John. The boy questions John to see if he is friend or foe. 

He doesn't answer John at first. 

The boy responds with a nice friendly attitude. 
» Q.4 . How do the boy's feelings for John change as the story jMOgresses? 

B The boy begins to talk more to John. He feels more comfortable with him and he confides in him. 

P But at the end he dislikes him again. 

_ He gr^ually starts trusting John and looks up to him. Then he feels betrayed when they go to the 

I police station and the boy despises John. 

Score 2 

I As the story progresses, the boy begins to think that John has betrayed his confidence. 

* The boy seems to trust John more and more throughout the story. 

Q.5. At the end of tl^ story why does John Deweck never again return to the fountain? 

He feels that he has once and for all beutiyed youth, 
gj Because he hurt the boy's feelings so much even though he did what was best for the boy. 

^ Score! 

• The boy left John wiili a bad feciing and John doesn't want to be left that way again. 
■ Score 1 

He doesn't want to feel grief over the death of his wife anymore. 

ERIC 



He is not in ihe city any longer. He has moved out of the city and started his life over. 

Q.6. What do the different descriptions of the fountain contribute to the development of the relationship 
betw^n John Deweck and the boy? 

Score 

As John and the boys relationship grows stronger and then not at alK the descriptions of the fountain 
change showing the danger to come. 

One notices the good things about life and the other notices the bad. This gives us our conflict. 

At first the relationship was good but then it started to change to a bad one. 
The way the fountain is makes John behave in a specific way. 

How they like each other maT and more. 
"The Use of Force'* 

Q. 1. Describe how Mathilda responds to the doctor at the beginning of the story. 
Score 3 

She sits there looking at him with distrust and suspicion in her eyes at first. Then she won't open her 
mouth for him to look at. 

The child seems uncaring but you can see that she hates him. She hated and feared him but wouldn't 
show it to anyone but the doctor. She didn't want her parents to know at first. 

She was quiet and did not move. She was afraid of the doctcM-. 

She was fairly eating him up with her cold shady eyes and no expression on her face whatever. 
Mathilda was sad and had no emotions or feelings towards anything in the beginning. 
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Q,2. How does the docior respond to Mathilda at the beginning of the story? 

He saw the girl with a strange beauty. He tried at first calmly to get her to open her mouth. 
The doctor was captivated by her. He speaks of her appearance reverently. 

He sees her as a nice little girl but feels she also was staring him down. 
He tries to talk nicely to her, to sweet talk her into opening up. 

He distrusts the family. 
Q.3. Why does Mathilda claw fw the doctor's eyes? 

The girl fell threatened by the do^:tor and became quite afraid when he moved closer to her, so she 
clawed at his eyes so he wouldn t lock down her throat. 

She is guarding and (m)tecting herself from the doctor. She fears him and she also fears death. 

Because the doctor told her he huil nothing in his hand when he had a splint. 
She doesn't like him and doesn I want him to touch her. 

Q.4. How do you think the doctor s altitude toward Mathilda changes as he proceeds with the 
examination? 

As the doctor proceeds he begins to hate her and wants to strangle t^r. 

As the docior proceeds with the exam his attitude of adoration and calmness changed as she fought him 
to dislike and fury. 

At first the doctw sees her as a nice but stubborn little girL but as he goes on he sees her as a brat. 
He gets disturbed and furious by her actions- when she won't let him help her. 
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He doesn't care that she will die. he hates her. 
Q.5. How would you describe the doctor's responses to ihe parenis? 

He seemed to dislike the parents* especially everylime they did something to help. He felt he was the 
doctor and Ik didn't need any of tteir annoying help. 

His responses to the parents were harsh. 

The doct(x thinks he can handle tt^ child and their comments aren't needed. 
The doctor's responses are that they are doing more bad than good in the room. 
Score 1 

He doesn't want them to be nice to her. 
Q.6. How does the doctor s p xepUon of himself change from the beginning to the end of the stoiy? 

The doctor sees himself getting angrier and wilder to get her mouth open no matter what* 

He begins to feel that he is a failure. He couldn't just leave her because she would die yet he couldn't 
win in this situation without using force. 

At the beginning of the story he wants to help but now it is more of a challenge to him. 

He sees himself winning. 
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telions" 

Q. i . Where do you think this baulcficld is? 
Score 3 

In someone's tmclcyard, they re ciihcr cutting the grass or weeds killer. 
In a neighbor's lawn where the dandelions are growing. 

I think the battlefield is a meadow, or what used to be one near a rural town. 
A minefield in an old (xchard or somcL*iing, 
Score 1 
Korea 

A War zone 

Q.2. Is there a victor in ihis baulc, who or what do you think it is? 

No. because the dandelions are starting lo take over the lawn and they don t do so successfully because 
people are spraying them with poison. 

There is no victor because even though the flov/ers are gone, they have allowed their remains to blow 
away to be recycled^ so to speak into new flowers. 

I don't think there is a victor but the landscape is losing. 

The eiwmy is the victor because it says at the end of die poem Utey will never surrender. 

Score 1 

The troops. 

The victor is the enemy because ihc author says he surrenders. 
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Q.3. What conflicis are represented by tte battles? 
The conflict is between humans and nature* 

The never ending war on weeds, they keep coming lack no matter what you do. 

The conflict of keeping a yard. 

The conflict is between good and bad. 



Q.4. What does each pow represent? 
Score 3 

Another blast of weed killer or growing energy. 

Everytime a new seed lands and is planted she uses the word "pow*'. 

Each pow represents a gun shot. 
Score i 

It was Vietnam, each pow is a fHisoner of war. 
Each pow represents death. 

Q.S. Describe the way the poet uses the progression of time in this poem. 

Score? 

Makes it seems tike it was only s^onds long when it really was days. 
The time is confusing, doesn't ^m to be the way time is really. 

He uses phrases like all day* all yesterday and all today. 
Score 1 

There is no set time, it is up to the readers imagination. 
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He describes what he sees while he is walking in a logical order 



Q.6. What senses are combined in the images in the poem? 
I Scored 

M The senses of hearing* seeing and feeling. What it's like in a battlefield or a lawn of dandelions are 

I combined in the images. 

Sounds and feeUny s of everything. 

"Forgive My Guilt" 

■ Q. 1 . From whom is the poem asking forgiveness? 

B From the other birds and wild animals, 

■ The poem asks forgiveness from "all that is wild* airy and beautiful/' 

I The hunter is asking the birds for forgiveness. 

The poem is asking you to forgive the boy who shot the birds, 

_ The poem is asking forgiveness from the person who is telling the story. 

■ Q. 2 . How are the birds' songs described? 
Score 3 

H The songs are described as slender flutes of souows. 

They are ''sorrowful high flul^** as described by the narrator. 

■ Score 2 

_ They are sad and sorrowful. 
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Q.3. What keeps the memory alive? 

The sounds of the birds still «:ho in his mind which constantly remind him. 

The songs the birds sang and the way it sounded, he hears this in his memory when he hears other 
sounds too. 

His guilt keeps the memory alive, he still feels bad. 

Seeing more birds flying in the sky. 

Q.5. What connections are there between the two stanzas of the poem? 

One tells what happened and the other how he feels now. 
One is about the incident, the other was after. 

The first tells of exactly what ha^Kned and the second tells of what he is reminded of. 
The two stanzas are connected by the thought of committing a sin. 

They say how he feels for shooting the birds. 



Q.6. What do you think the comparisons the poet makes about the wounded birds contribute to the 
poem? 

T\k birds cries were somewhat the samt as the guilty pain the boy felt. The birds were in iMin so was 
the boy. 

The poet uses comparisons very well to make you feel like he does. He makes the reader feel bad for 
the wounded birds and it makes you not want to do what he did. 
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They give ihc poem a more sorrowful affect 

The comparisons make the crime seem even worse and conveys the birds pain. 

Comparing how ihey used to fly south and now they can'L 
Imagery is about to take placx^ 

"Thelkii" 

Q.l . What happened to the Iks that made ihcm "celebrities'*? 

The govOTiment took their land and made ihcm farmers and also sent an anthropologic to this area for 
2 years and he wrote a book. 

The Iks are forced to change life style without any say in the situation. Their actions arc related to 
modem civilization in a book and this makes ihcm famous. 

A book was written about them that made them infamous* 

The Iks were looked at as a people who were forced to change completely. 

The Iks formed their own separate groups in which ihae was one Ike in that group in turn he was the 
only ruler. 

They were heartless and selfish, their land was taken away and an smf "^ropolc^ist wrote a book on ihem 
and told everyone how awful they were, 

Q.2. How do the narrator's ideas about the Iks differ from the anthropologist's ideas about them? 

The anthropologist feels that they arc not human beings but like animals who have no regard (a 
life at all. The narrator feels that ihcy did not choose to live as they now do but were made this way by the 
world around them. 

* 

The narrator doesn't believe the book is honest about the Dcs' life, it gives a distorted view and there are 
reasons for their behavior. The anthropologist tells it behind hateful eyes and exaggerates the jmAIcms of 
the Iks society. 
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The nairator views the Iks as trying lo be true to their culture whereas the anihropologisi views ihem 
as stubborn and unwilling to change. 

The narrator looks at the Iks on a comparative basis. The anthropologist seems only to be giving 
them a psychoanalysis* 

The narrator doesn't think they are people but the anthropologist does. 

QJ. What assumptions r,iust we agree with to accept the anthropologist's theory about the Iks? 
S ccMy 3 

We must agree that man was made to be bad and that he is out for himself only. Also graces like 
affectkxi and compassion are learned habits. 

We must assume that the Iks aie a new breed of cruel individuals and they are a totally insane society 
driven to this insanity by the problems their lives face. 

That if society were to break down we would all become Iks. 
The idea of it being right to change someone else's way of life. 

We must accept them as lazy, crazy, an overall waste of life. 
Q.4. Describe the organization of this essay. 

He first presented a history of the Iks. He then told the anthropologist's views of this and then told 
what he himself thought of the Iks and why. 

The essay is a mixture of a painting of a way of life and of Ik society along with two very opposite 
viewpoints. 

Ssas&2 

The organization is about how people changed and different viewpoints on the change that took place. 
A thesis-proof formal. Author opens with broad statement gives facts and closes with final opinion. 
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A story with good organization. Confusing at limes. The organizaiion is OK. The narrator displays 
his points in an orderly fashion, 

Q.5. How would you describe the author s attitude towards the anthropologi' ' 
Score 

The author clearly does not agree with the anthropologist and does not support his theory. 

The author seems to be laughing at or mocking the anthropologist by comparison of his theory with 
the anthropologists. 

He felt the anthropologist was ? fool basically. 
He fell sony for him. 

Q.6. What do the comparisons that the author makes between the Iks and social systems contribute to 
the way he presents his view of the Iks? 

He shows that the Iks were jroducts of our society. He feels that they became that way because society 
has shaped tt^m into this new self of theirs. 

It is obvious that the narrator thinks lowly of the social systen. 'hat make people act this way. 
That the Iks had no social system so they made their own. 

He feels that everyone should have the right to say how ihey feel and be what they want. 
To show what he thinks of the Iks, 
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"Birth ofihe Moon'* 

Q, 1 . Why is the Impacts Theory a favorite of asironomers? 
Score 1 

It helps explain our neighbor's very unusual chemistry. 

Because it explains why the moons* rocks are like the earth s rocks and whv ihev arc mn like the earth's 
rocks. 

Because tt may explain how the moon was ban some 4 and a half billion years ago. 
It suggests that the moon is in some way a piece of the eanh. 
Score 1 

Because they were interested in how the moon was bom. 

Q,2, What similarities and differences between the eanh and the moon are revealed by the moon rocks' 
fmgeiprint? 

The similarities are isotope proporttons of the rocks which contain magnesium, manganese and silicon. 
The differences are that earth has plenty of metallic iron and volatUes, the moon few. 

They both contain some of the same isotopes. 

Q3. Compare the way the astronomers qu«ed differ in atlinide toward the Impaclor Theory. 
Scored 

They had many different theories. Some say the moon was trapped inside the earth's gravity while 
zooming through space, or it split off from earth in a hot spinning cloud. They don't agree on what 
happened. 

The first astromnner seemed positive of the Impactor Theory, and the last one qiK>ted was doubtful. 
Scor^2 

One says, "no one knows what really happened**, another says, "the whole event took less than a day 
and a lot of material went into cMi aiound the earth.** 

They all believe that it could happen this way but some aren't sure. 
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QA. What do the wwds "impocior**, **whack'\ and "splash" have in common? 
These words all mean crash or collision, 

They all have th^ same idea that of 2 Ihings crashing together violently. 
S£QI£i 

They all are words ihat are used wiih the theory. They all have to do with touching something. 
SccceJL 

They are all used in the story. 



Q.5« Describe the way the Impactor Tliec^ 'i presented and explained in this essay, 
Score 3 

It is presented and explained so that the average person may understand it. Big words are used but 
explained so the meaning is understood 

Tt^ thec»7 is presented as a theory. But as the most plausible theory. At the end, I was still to decide 
whether to believe it or not using the infonnation. 

The theory is presented on how the moon came about by hot gas caused combustion. 

Billions of years ago a planet-like object cu^'ihed into the Earth. The object spht apart and eventually a 
piece of the object fomfied the moon. 



Q.6. How is the Impactor Theory present differently in the first and last paragraphs of this essay? 

It is explained in detail in paragraph I how it might have happened and in the last it was debated as if it 
was or wasn't the reason for the moon being created. 

In the first they say this is how it happened but the in last they say this is how it could have happened. 
One is a story and the other an opinion. 
They coincide with e»:h other. 
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Appendix 4 

Supplemental Tables: 
Test and Item Characteristics 



Table 1 

Short Answer Measures (Scale 0 — 18) 



Means Standard Deviation 



Story 1 
The Use of 
Force 

Story 2 
The Man by 
the Fountain 

Poem 1 
Forgive my 
Guiit 

Poem 2 
Dandelions 



15.96 



14.67 



14.83 



12.97 



3.24 



4.07 



3.17 



3.90 



Nonfiction 1 
The Birth of 
the Moon 

Nonfiction 2 
The Iks 



15.71 



13.98 



3.77 



4.84 
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Tabic 2 

T-Tesi for Shon Answer Measures across Six Texts 



STORY POEM NONFICnON 

12 12 12 



Story 1 

The Use of 2.93* 3.00* 6.98* .58 4.03* 
Force 



Story 2 

The Man by -.37 3.57* -2.24* 1.30 
the Fountain 

Poem 1 

Forgive my 4.45* -2.18* 1.78 
Guilt 

Poem 2 

Dandelions -6.03* -1.93 



Nonfiction 1 

The Birth of 3.38* 
the Moon 

Nonfiction 2 
The Iks 



* P< .05 
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• Table 3 

Short Answer Measures (Scale 0 — 3) 



ITEM DIFFICULTY 
TEXT/ITEMS 1 2 3 4 5 6 



Siory I 

The Use of 2.76 2.76 2.74 2.73 2.59 2.39 

Force 



Siory 2 

The Man by 2.28 2.73 2.75 2.55 2.34 2.07 

the Fountain 



Poem 1 

Forgive my 2.52 2.76 2.51 2.76 2.21 2.05 

Guilt 



Poem 2 

Dandelions 2.28 2.33 1.99 2.18 1.99 2.21 



Nonflction 1 

The Birth of 2.78 2.85 2.36 2.74 2.51 2.48 

the Moon 



Nonflction 2 

The Iks 2.52 2.53 2.33 2.13 2.36 2.11 



M 
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Table 4 
Essay (Scale 0 — 3} 



Means Siaiidard Deviation 



Story I 

The Use of 1.96 .88 

Force 



Story 2 

The Man by 1.69 1.18 

the Fountain 



Poem I 

Forgive my 2 35 .83 

Guilt 



Poem 2 

Dandelions 1.16 1.05 



Nonficiion 1 

TheBinhof 2,14 1. 00 

the Moon 



Nonfiction 2 

The Iks 1.90 1.15 
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Table 5 

T-Test for Essay across Six Texts 



STORY POEM NONFICTION 

12 12 12 



Story I 

The Use of 2.10* -3.64* 2.93* -1.51 .48 
Force 

Story 2 

The Man by -5.27* .60 3.38* -1.46 

the Fountain 

Poem 1 

Forgive my 6.38* 1.88 3.62* 

Guilt 



the Moon 



Nonfiction 2 
The Iks 

• P< .05 



Poem 2 

Dandelions -4.25* -2.15* 



Nonfiw*'on 1 

TheBinnnf 1.80 
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Table 6 



Aesthetic Criteria Choice among 10* 







Eoumaia 


Guilt 




Mqqh 


Iki 


True to life 


17% 


8% 


8% 


9% 


12% 


13% 


Oi^anization 


5% 


4% 


8% 


8% 


22% 


18% 


Impact 


32% 


19% 


40% 


27% 


10% 


12% 


M(xal 


10% 


17% 


12% 


1% 


6% 


12% 


Style 


4% 


11% 


6% 


22% 


19% 


13% 


Originality 


8% 


10% 


3% 


10% 


8% 


7% 


Social Relevance 


2% 


8% 


1% 


1% 


4% 


7% 


Engagement 


18% 


17% »- 


12% 


14% 


10% 


11% 


Metaphors 


2% 


8% 


7% 


8% 


9% 


6% 


Personal Rel- 


2% 


0% 


2% 


1% 


1% 


1% 
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Table 7 

Mean Rating of Quality (Scale 0 — 6) 



Standard 
Mean Deviation 

Story I 

The Use of 4.10 1.33 

Force 



Story 2 

The Man by 4.00 1.32 

the Fountain 



I^m 1 

Forgive my 4.50 1.09 

Guilt 



Poem 2 

Dandelions 3.66 1.34 



Nonfiction 1 

TheBinhof 3.97 1.36 

the Moon 



Nonfiction 2 

The DCS 3.83 1.41 
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Table 8 



Background (Cultural Information) 
Scales 1 14 



MSim Standard Deviation 

5.67 3.72 



Item Difficulty (In percentage) 



1 2 3 4 S 6 7 8 2 IQ U 12 L2 L4 

22.9 70.6 8.9 66.9 18.4 48.5 64.8 37.6 35.4 52.7 24.7 32.3 67.0 17.6 



Table 9 

Background (Terms) 
Scale 0 — 30 



Mean Standard Deviation 

18.19 8.63 



Item Difficulty { 0 — 3 ) 

1 2 3 4 5 6 2 8 9 IQ 

1.76 2.15 1.14 1.86 1.70 2.47 1.96 1.93 1.23 1.99 



Table 10 
Attitu(k Transfer and Interest 



Transfer Interest 

(SCALES 0 . 40) (SCALES 0 -36) 

^IS^ Standard Deviation M^aos Standard Peviatlon 

24.21 5.50 18.66 4.97 
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Table 1 1 
Censorship (S'^oje 0 --96) 



MeaUS Standard Dgviaiinns 
47.48 10,90 



Table 12 
Responses to Source of Reading 



When you choose a siory of novel to read, which one of rhe following 
is most likely to be the reason for your choice? 



RESPONSES FREQUENCY PERCENTAGE 

a. Friends or parents recommend ii 152 17.7 



b. I have read other books by the 






same author 


260 


30.3 


c. The title attracts me 


272 


31.7 


d. I just choose any 


61 


7.1 



Table 13 

Item Scores on Knowledge of Literary Teims (Range 0-3) 
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Number 


Mean 








Peviation 


1 


1.76 


1.22 


Am 


2.15 


1.07 


3 


1,14 


1.31 


4 


1.86 


1.33 


5 


1.70 


1.32 


6 


2.47 


0.93 


7 


1.96 


1.09 


8 


1.93 


1.26 


9 


1.23 


1.21 


10 


1.99 


1.31 




56 





Table 14 



Item Result); Mythologv und Legend Questions 



Item 


Prngniase 






1 


22.9 


2 


70.6 


3 


8.9 


4 


66.9 


5 


18.4 


6 


48.5 


7 


M.S 


8 


37.6 


9 


35.4 


10 


52.7 


11 


24.7 


12 


32.3 


13 


67.0 


14 


17.6 
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